<!DOCTYPE html>
<html lang="en">
<head profile="http://a9.com/-/spec/opensearch/1.1/">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="../assets/site.css" rel="stylesheet">
  <title>go/scanner</title>
  
    <meta name="twitter:title" content="Package scanner">
    <meta property="og:title" content="Package scanner">
    <meta name="description" content="Package scanner implements a scanner for Go source text.">
    <meta name="twitter:description" content="Package scanner implements a scanner for Go source text.">
    <meta property="og:description" content="Package scanner implements a scanner for Go source text.">
    <meta name="twitter:card" content="summary">
    <meta name="twitter:site" content="@golang">
  
  

  

</head>
<body>

<div class="container">

        <h2 id="pkg-overview">package scanner</h2>

        <p><code>import "go/scanner"</code>

        <p>
Package scanner implements a scanner for Go source text.
It takes a []byte as source which can then be tokenized
through repeated calls to the Scan method.</p>


        
  


        
        <h3 id="pkg-index" class="section-header">Index <a class="permalink" href="#pkg-index">&para;</a></h3>

        

        <ul class="list-unstyled">
          
          
          <li><a href="#PrintError">func PrintError(w io.Writer, err error)</a></li>
          
            <li><a href="#Error">type Error</a></li>
            <ul>
            
            <li><a href="#Error.Error">func (e Error) Error() string</a></li>
            </ul>
          
            <li><a href="#ErrorHandler">type ErrorHandler</a></li>
            
            
            
            
          
            <li><a href="#ErrorList">type ErrorList</a></li>
            <ul>
            
            <li><a href="#ErrorList.Add">func (p *ErrorList) Add(pos token.Position, msg string)</a></li><li><a href="#ErrorList.Err">func (p ErrorList) Err() error</a></li><li><a href="#ErrorList.Error">func (p ErrorList) Error() string</a></li><li><a href="#ErrorList.Len">func (p ErrorList) Len() int</a></li><li><a href="#ErrorList.Less">func (p ErrorList) Less(i, j int) bool</a></li><li><a href="#ErrorList.RemoveMultiples">func (p *ErrorList) RemoveMultiples()</a></li><li><a href="#ErrorList.Reset">func (p *ErrorList) Reset()</a></li><li><a href="#ErrorList.Sort">func (p ErrorList) Sort()</a></li><li><a href="#ErrorList.Swap">func (p ErrorList) Swap(i, j int)</a></li>
            </ul>
          
            <li><a href="#Mode">type Mode</a></li>
            
            
            
            
          
            <li><a href="#Scanner">type Scanner</a></li>
            <ul>
            
            <li><a href="#Scanner.Init">func (s *Scanner) Init(file *token.File, src []byte, err ErrorHandler, mode Mode)</a></li><li><a href="#Scanner.Scan">func (s *Scanner) Scan() (pos token.Pos, tok token.Token, lit string)</a></li>
            </ul>
          
          
        </ul>

        
        
          <h4 id="pkg-examples">Examples <a class="permalink" href="#pkg-examples">&para;</a></h4>
          <ul class="list-unstyled">
            <li><a href="#example-Scanner-Scan" onclick="$('#ex-Scanner-Scan').addClass('in').removeClass('collapse').height('auto')">Scanner.Scan</a></li>
          </ul>
        

        
        
<h4 id="pkg-files">
  <a href="https://github.com/golang/go/blob/master/src/go/scanner/">Package Files</a>
  <a class="permalink" href="#pkg-files">&para;</a>
</h4>

<p><a href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go">errors.go</a> <a href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go">scanner.go</a> </p>

        
        

        
        

        
        
        
          <h3 id="PrintError" data-kind="f">func <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L116">PrintError</a> <a class="permalink" href="#PrintError">&para;</a> <a class="uses" title="List Function Callers" href="https://sourcegraph.com/-/godoc/refs?def=PrintError&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h3>
          <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L116">❖</a><pre>func PrintError(w <a href="/io">io</a>.<a href="/io#Writer">Writer</a>, err <a href="/builtin#error">error</a>)</pre></div><p>
PrintError is a utility function that prints a list of errors to w,
one error per line, if the err parameter is an ErrorList. Otherwise
it prints the err string.
</p>

          
  

        

        
        

        
          <h3 id="Error" data-kind="t">type <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L19">Error</a> <a class="permalink" href="#Error">&para;</a> <a class="uses" title="List Uses of This Type" href="https://sourcegraph.com/-/godoc/refs?def=Error&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h3>
          <div class="decl" data-kind="d"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L19">❖</a><pre>type Error struct {
    <span id="Error.Pos">Pos</span> <a href="/go/token">token</a>.<a href="/go/token#Position">Position</a>
    <span id="Error.Msg">Msg</span> <a href="/builtin#string">string</a>
}</pre></div><p>
In an ErrorList, an error is represented by an *Error.
The position Pos, if valid, points to the beginning of
the offending token, and the error condition is described
by Msg.
</p>

          
          
          
  


          

          
            <h4 id="Error.Error" data-kind="m">func (Error) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L25">Error</a> <a class="permalink" href="#Error.Error">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=Error%2FError&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L25">❖</a><pre>func (e <a href="#Error">Error</a>) Error() <a href="/builtin#string">string</a></pre></div><p>
Error implements the error interface.
</p>

            
  

          
        
          <h3 id="ErrorHandler" data-kind="t">type <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L26">ErrorHandler</a> <a class="permalink" href="#ErrorHandler">&para;</a> <a class="uses" title="List Uses of This Type" href="https://sourcegraph.com/-/godoc/refs?def=ErrorHandler&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h3>
          <div class="decl" data-kind="d"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L26">❖</a><pre>type ErrorHandler func(pos <a href="/go/token">token</a>.<a href="/go/token#Position">Position</a>, msg <a href="/builtin#string">string</a>)</pre></div><p>
An ErrorHandler may be provided to Scanner.Init. If a syntax error is
encountered and a handler was installed, the handler is called with a
position and an error message. The position points to the beginning of
the offending token.
</p>

          
          
          
  


          

          
        
          <h3 id="ErrorList" data-kind="t">type <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L37">ErrorList</a> <a class="permalink" href="#ErrorList">&para;</a> <a class="uses" title="List Uses of This Type" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h3>
          <div class="decl" data-kind="d"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L37">❖</a><pre>type ErrorList []*<a href="#Error">Error</a></pre></div><p>
ErrorList is a list of *Errors.
The zero value for an ErrorList is an empty ErrorList ready to use.
</p>

          
          
          
  


          

          
            <h4 id="ErrorList.Add" data-kind="m">func (*ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L40">Add</a> <a class="permalink" href="#ErrorList.Add">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FAdd&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L40">❖</a><pre>func (p *<a href="#ErrorList">ErrorList</a>) Add(pos <a href="/go/token">token</a>.<a href="/go/token#Position">Position</a>, msg <a href="/builtin#string">string</a>)</pre></div><p>
Add adds an Error with given position and error message to an ErrorList.
</p>

            
  

          
            <h4 id="ErrorList.Err" data-kind="m">func (ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L105">Err</a> <a class="permalink" href="#ErrorList.Err">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FErr&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L105">❖</a><pre>func (p <a href="#ErrorList">ErrorList</a>) Err() <a href="/builtin#error">error</a></pre></div><p>
Err returns an error equivalent to this error list.
If the list is empty, Err returns nil.
</p>

            
  

          
            <h4 id="ErrorList.Error" data-kind="m">func (ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L93">Error</a> <a class="permalink" href="#ErrorList.Error">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FError&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L93">❖</a><pre>func (p <a href="#ErrorList">ErrorList</a>) Error() <a href="/builtin#string">string</a></pre></div><p>
An ErrorList implements the error interface.
</p>

            
  

          
            <h4 id="ErrorList.Len" data-kind="m">func (ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L48">Len</a> <a class="permalink" href="#ErrorList.Len">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FLen&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L48">❖</a><pre>func (p <a href="#ErrorList">ErrorList</a>) Len() <a href="/builtin#int">int</a></pre></div><p>
ErrorList implements the sort Interface.
</p>

            
  

          
            <h4 id="ErrorList.Less" data-kind="m">func (ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L51">Less</a> <a class="permalink" href="#ErrorList.Less">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FLess&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L51">❖</a><pre>func (p <a href="#ErrorList">ErrorList</a>) Less(i, j <a href="/builtin#int">int</a>) <a href="/builtin#bool">bool</a></pre></div>
            
  

          
            <h4 id="ErrorList.RemoveMultiples" data-kind="m">func (*ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L78">RemoveMultiples</a> <a class="permalink" href="#ErrorList.RemoveMultiples">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FRemoveMultiples&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L78">❖</a><pre>func (p *<a href="#ErrorList">ErrorList</a>) RemoveMultiples()</pre></div><p>
RemoveMultiples sorts an ErrorList and removes all but the first error per line.
</p>

            
  

          
            <h4 id="ErrorList.Reset" data-kind="m">func (*ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L45">Reset</a> <a class="permalink" href="#ErrorList.Reset">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FReset&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L45">❖</a><pre>func (p *<a href="#ErrorList">ErrorList</a>) Reset()</pre></div><p>
Reset resets an ErrorList to no errors.
</p>

            
  

          
            <h4 id="ErrorList.Sort" data-kind="m">func (ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L73">Sort</a> <a class="permalink" href="#ErrorList.Sort">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FSort&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L73">❖</a><pre>func (p <a href="#ErrorList">ErrorList</a>) Sort()</pre></div><p>
Sort sorts an ErrorList. *Error entries are sorted by position,
other errors are sorted by error message, and before any *Error
entry.
</p>

            
  

          
            <h4 id="ErrorList.Swap" data-kind="m">func (ErrorList) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L49">Swap</a> <a class="permalink" href="#ErrorList.Swap">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=ErrorList%2FSwap&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/errors.go#L49">❖</a><pre>func (p <a href="#ErrorList">ErrorList</a>) Swap(i, j <a href="/builtin#int">int</a>)</pre></div>
            
  

          
        
          <h3 id="Mode" data-kind="t">type <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L91">Mode</a> <a class="permalink" href="#Mode">&para;</a> <a class="uses" title="List Uses of This Type" href="https://sourcegraph.com/-/godoc/refs?def=Mode&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h3>
          <div class="decl" data-kind="d"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L91">❖</a><pre>type Mode <a href="/builtin#uint">uint</a></pre></div><p>
A mode value is a set of flags (or 0).
They control scanner behavior.
</p>

          <div class="decl" data-kind="c"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L93">❖</a><pre>const (
    <span id="ScanComments">ScanComments</span> <a href="#Mode">Mode</a> = 1 &lt;&lt; <a href="/builtin#iota">iota</a> <span class="com">// return comments as COMMENT tokens</span>

)</pre></div>
          
          
  


          

          
        
          <h3 id="Scanner" data-kind="t">type <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L32">Scanner</a> <a class="permalink" href="#Scanner">&para;</a> <a class="uses" title="List Uses of This Type" href="https://sourcegraph.com/-/godoc/refs?def=Scanner&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h3>
          <div class="decl" data-kind="d"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L32">❖</a><pre>type Scanner struct {

    <span class="com">// public state - ok to modify</span>
    <span id="Scanner.ErrorCount">ErrorCount</span> <a href="/builtin#int">int</a> <span class="com">// number of errors encountered
    // contains filtered or unexported fields</span>
}</pre></div><p>
A Scanner holds the scanner&#39;s internal state while processing
a given text. It can be allocated as part of another data
structure but must be initialized via Init before use.
</p>

          
          
          
  


          

          
            <h4 id="Scanner.Init" data-kind="m">func (*Scanner) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L113">Init</a> <a class="permalink" href="#Scanner.Init">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=Scanner%2FInit&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L113">❖</a><pre>func (s *<a href="#Scanner">Scanner</a>) Init(file *<a href="/go/token">token</a>.<a href="/go/token#File">File</a>, src []<a href="/builtin#byte">byte</a>, err <a href="#ErrorHandler">ErrorHandler</a>, mode <a href="#Mode">Mode</a>)</pre></div><p>
Init prepares the scanner s to tokenize the text src by setting the
scanner at the beginning of src. The scanner uses the file set file
for position information and it adds line information for each line.
It is ok to re-use the same file when re-scanning the same file as
line information which is already present is ignored. Init causes a
panic if the file size does not match the src size.
</p>
<p>
Calls to Scan will invoke the error handler err if they encounter a
syntax error and err is not nil. Also, for each error encountered,
the Scanner field ErrorCount is incremented by one. The mode parameter
determines how comments are handled.
</p>
<p>
Note that Init may call err if there is an error in the first character
of the file.
</p>

            
  

          
            <h4 id="Scanner.Scan" data-kind="m">func (*Scanner) <a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L602">Scan</a> <a class="permalink" href="#Scanner.Scan">&para;</a> <a class="uses" title="List Method Callers" href="https://sourcegraph.com/-/godoc/refs?def=Scanner%2FScan&amp;pkg=go%2Fscanner&amp;repo=">Uses</a></h4>
            <div class="funcdecl decl"><a title="View Source" href="https://github.com/golang/go/blob/master/src/go/scanner/scanner.go#L602">❖</a><pre>func (s *<a href="#Scanner">Scanner</a>) Scan() (pos <a href="/go/token">token</a>.<a href="/go/token#Pos">Pos</a>, tok <a href="/go/token">token</a>.<a href="/go/token#Token">Token</a>, lit <a href="/builtin#string">string</a>)</pre></div><p>
Scan scans the next token and returns the token position, the token,
and its literal string if applicable. The source end is indicated by
token.EOF.
</p>
<p>
If the returned token is a literal (token.IDENT, token.INT, token.FLOAT,
token.IMAG, token.CHAR, token.STRING) or token.COMMENT, the literal string
has the corresponding value.
</p>
<p>
If the returned token is a keyword, the literal string is the keyword.
</p>
<p>
If the returned token is token.SEMICOLON, the corresponding
literal string is &#34;;&#34; if the semicolon was present in the source,
and &#34;\n&#34; if the semicolon was inserted because of a newline or
at EOF.
</p>
<p>
If the returned token is token.ILLEGAL, the literal string is the
offending character.
</p>
<p>
In all other cases, Scan returns an empty literal string.
</p>
<p>
For more tolerant parsing, Scan will return a valid token if
possible even if a syntax error was encountered. Thus, even
if the resulting token sequence contains no illegal tokens,
a client may not assume that no error occurred. Instead it
must check the scanner&#39;s ErrorCount or the number of calls
of the error handler, if there was one installed.
</p>
<p>
Scan adds line information to the file added to the file
set with Init. Token positions are relative to that file
and thus relative to the file set.
</p>

            
  
    <div class="panel-group">
    
      <div class="panel panel-default" id="example-Scanner-Scan">
        <div class="panel-heading"><a class="accordion-toggle" data-toggle="collapse" href="#ex-Scanner-Scan">Example</a></div>
        <div id="ex-Scanner-Scan" class="panel-collapse collapse"><div class="panel-body">
          
          <p>Code:<span class="pull-right"><a href="?play=Scanner-Scan">play</a>&nbsp;</span>
          <pre><span class="com">// src is the input that we want to tokenize.</span>
src := []byte(&#34;cos(x) + 1i*sin(x) // Euler&#34;)

<span class="com">// Initialize the scanner.</span>
var s scanner.Scanner
fset := token.NewFileSet()                      <span class="com">// positions are relative to fset</span>
file := fset.AddFile(&#34;&#34;, fset.Base(), len(src)) <span class="com">// register input &#34;file&#34;</span>
s.Init(file, src, nil <span class="com">/* no error handler */</span>, scanner.ScanComments)

<span class="com">// Repeated calls to Scan yield the token sequence found in the input.</span>
for {
    pos, tok, lit := s.Scan()
    if tok == token.EOF {
        break
    }
    fmt.Printf(&#34;%s\t%s\t%q\n&#34;, fset.Position(pos), tok, lit)
}</pre>
          <p>Output:<pre>1:1	IDENT	&#34;cos&#34;
1:4	(	&#34;&#34;
1:5	IDENT	&#34;x&#34;
1:6	)	&#34;&#34;
1:8	&#43;	&#34;&#34;
1:10	IMAG	&#34;1i&#34;
1:12	*	&#34;&#34;
1:13	IDENT	&#34;sin&#34;
1:16	(	&#34;&#34;
1:17	IDENT	&#34;x&#34;
1:18	)	&#34;&#34;
1:20	;	&#34;\n&#34;
1:20	COMMENT	&#34;// Euler&#34;
</pre>
        </div></div>
      </div>
    
    </div>
  

<div id="x-footer" class="clearfix">
  <div class="container">
    <a href="http://studygolang.com/" target="_blank">Go语言中文网</a>
    <span class="text-muted">|</span> <a href="http://golang.org/" target="_blank">Go Language</a>
    <span class="pull-right"><a href="#">Back to top</a></span>
  </div>
</div>
<script src="../assets/jquery-2.0.3.min.js"></script>
<script src="../assets/bootstrap.min.js"></script>
<script src="../assets/site.js"></script>
</body>
</html>
